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ABSTRACT 

This paper suggests that considering fundamental 
issues in educational research may help researchers conceptualize 
more productive research methodologies for computer based instruction 
(CBI). inconsistent findings and methodological shortcomings in media 
research, including CBI, raise the possibility that current research 
methodologies may be inadequate to significantly enhance our 
understanding of CBI. The purpose of this document is to highlight a 
set of conceptual and methodological considerations which may be 
relevant to a comparison of goodness of fit between research 
methodology and the types of questions for which researchers seek 
answers. An examination of these issues leads to specific 
recommendations for conceptualizing and conducting experiments 
involving the computer in instruction. Specific recommendations are 
discussed in the following categories: moving towards theory 
building, expanding research methodologies, and setting priorities 
for applied research. The need for inclusion of each category is 
explained and justified, and some common misconceptions about each 
are allayed. A list of references is include.*!. (Author/JB) 
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Abstract 

Current research methodologies may be inadequate to 
significantly enhance our understanding of computer-based 
instruction (CBI) . Inconsistent findings and methodological 
shortcomings in media research, including CBI, raise ' this 
possibility. .This paper suggests that considering fundamental 
issues in educational research may help researchers 
conceptualize more productive research methodologies for CBI. 
An examination of these issues should lead to specific 
recommendations for conceptualising and conducting experiments 
involving the computer in instruction. Specific recommendations 
aro discussed in the following categories: moving towards theory 
building, expanding research methodologies, and setting 
priorities for applied research. 
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On Conceptua lizing Research Methodologies 
for Computer- Based Instruction 

The pitfalls of educational research are legion. The risks 
of succumbing to these pitfalls may be amplified when research 
involves innovative technologies like the computer. The 
exciting potential of the computer for enhancing instruction 
generates spontaneous enthusiasm and a concomitant need to 
justify this enthusiasm. Consequently , there may be a 
temptation to short-circuit the research process by asking 
superficial questions, by sacrificing sound methodology and by 
over-generalizing findings. The history of research 

investigating a variety of instructional media also suggests 
caution for those interested in researching topics in 
computer-based instruction (CBI) , Several writers have 
chronicled the failure of researchers to create a useful 
research base for .guiding the selection and use of instructional 
media (Clark and Bo vy , 1983; Di Vesta, 1975 ; Jamison, Suppes , and 
Wells, 1974; Leifer, 1976; Oettinger and Zapol , 1971; Saettler, 
1968) . 

Kxisting CBI research appears to fare no better. Evidence 
that methodology, for example, is a serious concern can be found 
in Ku 1 i k 1 s (1 983) meta-analysis of CBI studies in which a 
majority of the available studies were eliminated due to 
methodological shortcomings. Before we accept the possibility 
that strong empirical evidence is not possible or alternatively 
continue to muddy the empirical waters, we should compare the 
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CONCEPTUAL I Z IMG RESEARCH METHODOLOGIES 
goodness of fit between research methodology and the types of 
questions for which we seek answers. The purpose of this paper 
is to highlight a set of conceptual- and methodological 
considerations which may be relevant to/ such a comparison. 

Fundamental Conceptual and Methodological Concerns 

Research methodology is presumably a function of the types 
of questions the researcher chooses to address. A 
conceptualization of a research study begins, therefore, with a 
question in the mind of the researcher. The old saw about 
getting a good answer only if there is a good question is 
important at this stage but the process is more complex when put 
into the context of educational research. The researcher must 
also consider potential answers to the question, how those 
answers might be explained, and use these notions to formulate a 
methodology for research. The challenge of research is not 
simply generating good questions, but evolving strategies for 
generating a limited set of answers. 

Put another way, the fundamental issue is whether or not 
research methodology will permit what Piatt (1968) has termed 
strong inferences as opposed to weak generalizations. Strong 
inferences are, of course, preferred but are similarly more 
difficult to ferret out. They demand careful control of 
variables which sometimes means a movement towards 11 laboratory" 
as opposed to "real world" conditions. Achieving strong 
inferences is also facilitated by the presence of a guiding 
theory. A theory aids In the generation of testable hypotheses, 
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expedites methodological decisions, and also serves as a bench 
mark for interpreting the data gathered. A theory enables 
experimental results to be inte.rpretted as "a case in point" as 
opposed to an isolated phenomenon with many alternative 
explanations. Without a theoretical perspective isolating a set 
of significant variables for study becomes difficult. 

As an example consider the following question which is 
typical of those .motivating existing CBI research: Can a 
computer help students learn concepts in biology? Without 
theoretical guidance or a concern for the level at which 
findings might be generalized a conceptual ly simple experiment 
may emerge from this question. A biological concept is selected 
and taught' via a computer to an appropriate popul ition after 
which some achievement measure is employed to compare these 
subjects to others taught the same concept via alternative 
media. The best a researcher can do using this methodology is 
to report results and speculate b:' ■ ily as to what may have 
caused them. At worst, a misguided or over-enthusiastic 
researcher will on occasion use the resulting "ata to make 
general statements concerning the usefulness of computers to 
teach biological concepts. 

Raw empiricism is the dominant characteristic of this type 
of study and many of the benefits of experimental research are 
lost or risk being perverted. Vc^ d kittle is contributed to an 
overall understanding of the computer in instruction or f o : that 
matter its utility in teaching biological concepts beyond the 
specific conditions of the experiment. Methodology in this case 
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CONCEPTUALIZING RESEARCH METHODOLOGIES 
has been conceived as only a logical extension of the question 
without considering the realm of possible answers. Current 
conceptualizations of CBI research methodology may suffer from 
this fundamental weakness. 

Specific Conceptual and Methodological Recommendations 

Even a casual review of CBI research indicates a tendency 
to duplicate the methodologies of other media research with 
little reason to believe that results will be any more 
enlightening. One way to address this concern is to re-examine 
the conceptual and methodological f o jndat ions of research 
involving instructional media. To be valuable, however, this 
exercise must result in specific recommendations which can guide 
the researcher. Below is an a 1 1 empir^T6"m o v e in this direction. 
Recommendations have been grouped into three broad categories: 
moving towards theory building, expanding research 
methodologies, and setting priorities for applied research. 

Moving Towards Theory Building \ 

\ I 

Laehenmeyer (1970) has argued that the predominant vie**' <jf 

// \ 

experimentation in the social sciences has been inefficient ^n 
that it does not facilitate the formulation of general theories. 

The typical research study in psychology and education 

/ 

investigates isolated hypotheses which are generated primar/ily 
from a review of previous research as opposed to observable 
phenomena. Presumably, the emulative effect of ^nany 
experiments will be the development of a general theory v^hich 
consolidates findings into a unified whole. This has rarely , if 
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CONCEPTUALIZING RESEARCH METHODOLOGIES 
ever, occurred, however, in .educational research. 

Most successful theory building, on the other hand, occurs 
when general principles are inducted from directly observable, 
naturally occurring phenomena. The initial goal of the 
researcher is to develop adequate measurement instruments and 
methodologies for studying a readily observable phenomenon. 
Later, after fact finding pilot studies, a theoretical statement, 
may emerge to explain facts related to the phenomena studied. At 
this point, the theoretical statement can be used deductively to 
generate hypotheses for experimental verification. 

The need for developing a theoretical orientation to media 
research has been articulated by a number of writers. Salomon 
(1979) and Salomon and Clark (1977), for example, have 
attributed the lack of consistent findings in media research to 
the absence of a theoretical orientation. They feel that most 
of the existing research is a result, in their words, of 
investigations wi th media to determine instructional 
effectiveness. Instead, they propose that a theoretical 
oreintation leads to research on media to determine 
psychological effects . Th is puts che investigation of 
instructional media like the computer within the realm of 
psychological and educational theory instead of existing in a 
theoretical vacuum. Similarly, Clark and Bovy (1983) hav£ 
cautioned against the confounding pf technology and 
instructional method in formulating and conducting research 
involving instructional media. Ellis (1976) presaged these 
positions when he argued that understanding the use o-f computers 
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in education real ly begins with an understanding of education. 

A move towards a theory building orientation in CBI 
research would have significant impact on research 
methodologies. More studies would be conceived, carried out, 
and interpretted as pilot or exploratory studies. The 
importance of studies which focus on sharpening measurement 
instruments and developing workable methodologies would be 

i 

recognized. In addition, the likelihood that findings would be 
over-generalized would be reduced. In short, even the roost 
applied research would be "judged in terms of its contribution to 
a broader understanding ' of the ' computer 1 s role as a medium of 
instruction. 

At the same time focusing on theory building would provide 
a rationale for basic research in CBI. Basic research, for 
example, is possible when the computer is integrated into a more 
general theory of instructional media. Theoretical positions 
like those developed by Salomon (1979) and Olson (1976) could be 
used to gu ide the conceptualization of basic research hypotheses 
and the development of suitable metnodolog ies . By carrying out 
basic research founded on accepted theoretical positions 
researchers would be forced to confront and tease out the 
significant attributes of the computer as a medium of 
instruction. We would see fewer studies which make 
recommendations as to whether the computer is a viable medium 
for instruction and perhaps more that would give us insight into 
when, where, arid with whom the computer might best be suited for 
particular categories of instructional content. 
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Expanding Research Methodologies 

Building theory implies a search for those variables which 
may account for or predict the occurrence of a range of 
phenomena. When dealing with human behavior and learning, the 
researcher is assured that the number of variables and their 
interactions will be enormously complex. While a theory will 
reduce the number of variables to consider relevant for study, 
considerable complexity remains. When in-vestigat ing 

instructional uses of- the computer, identifying relevant 
variables becomes even more difficult. Computer technology 
makes the options for delivering instruction so open ended that 
even the set of possible variables may not be intuitively 
obvious (Reinking, 1984) . 

What does this imply specifically for research involving 
computers in instruction? In general this means expanding 
methodology to recognize a wider range of variables and to 
permit those which are significant to surface. Methodologies 
which recognize and pursue only technological differences may be 
too simplistic. Few useful inferences can be made about the 
computer 1 s relationship to instruction when so many potentially 
significant variables are ignored. 

A desire to contend with more variables miy dictate the 
need for more complex methodologies in experimental research, h 
wider range of statistical techniques may be necessary; 
multivariate and regression analyses may need to supplement 
univariate ANOVAs ♦ Even simple designs and straightforward 
statistical analyses, however, can be mad<e more powerful by 
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CONCEPTUALIZING RESEARCH METHODOLOGIES 
including multiple dependent measures and attempting to 
replicate results. The latter option will, of course, require 
the cooperation of journal editors. The experimental bias of 
educational and psychologic .1 research may also need to be 
tempered in order to encourage non-experimental methods 
employing correlational analyses or even case studies. Caution 
may also be necessary in foreclosing areas of inquiry on the 
basis of accepting a single null hypothesis. 

Examples of how creative research methodologies could 
proceed from theoretical underpinnings and a concern for 
relevant variables can be found in Salomon and Clark's (1977j 
examination of research methodologies for media research. They 
have offered several viable research designs and statistical 
techniques which would be directly applicable to CBI research. 

More productive methodologies may also be a result of 
collaborative research which brings together colleagues of 
varying expertise, The interacting variables which operate 
simultaneously when u^ing the computer in instruct: on can rarely 
be seen clearly by one individual. Ideally expertise would be 
sought out to insure knowledge of media, the instructional 
content and its pedagogy, and cognitive psychology. Even these 
areas do not exhaust all of the sources of significant variation 
(eg . social and environmental factors) . 

Setting Priorities for Appl ied Research 

A perhaps healthy tension has always existed between basic 
and applied research. Under ideal circumstances these research 
emphases complement each other so that both theory and practice 

ii 



CONCEPTUALIZING. RESEARCH METHODOLOGIES 
are mutually enriched. Under less than ideal circumstances the 
distinction between the two is unclear and research proceeds 
haphazardly with little clear . di rection. One of the theses of 
this paper is that the latter condition is more characteristic 
of media research in general and CBI research in particular. 

A common misconception, however, is that basic research is 
uniformly more essential to the theory building process. 
Lachenmeyer (1970) has cited several examples from the history 
of science which suggests that the opposite is more likely the 
case. Theory building . is at first more often a result of 
observing events in the real world which is the domain of the 
applied researcher. The applied researcher, however, must 
recognize the unique contribution applied research can make to 
theory building and conceptualize methodologies accordingly. 

This means more than suggesting theoretical explanations 
for findings; it also means seeking out real instructional 
problems that may contribute to a broader theoretical 
understanding of CBI. Wilkinson (1984) has suggested priorities 
for selecting research topics to explore computer applications 
in the teaching of a content area. First, the computer should 
be used to manipulate content in a fashion which is not readily 
duplicated by other instructional media. Secondly, the use of 
the computer should be guided by accepted pedagogical principles 
for teaching the content selected. Finally, a high priority 
should be given to research which addresses areas of instruction 
which have proven to be problematic. 

Researchers who subscribe to these priorities will increase 
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CONCEPTUALIZING RESEARCH METHODOLOGIES 
the likelihood that their research will be more relevant to the 
practitioner as well as contributing to the theory building 
process. 

Conclus ion 

In summary, current research methodologies ir.ay * be 
inadequate to significantly enhance our understanding of CBI • 
Inconsistent findings in media research, including CBI , raise 
this possibility. Considering fundamental issues in educational 
research suggests a broader range of research methodologies may 
be appropriate. The goal of CBI research should probably be 
identifying significant variables in an effort to build 
theoretical perspectives. Finally, the topics of applied 
research may need to be evaluated closely to insure their 
usefulness for both practice and theory development. 
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